Quantization for Signal Detection and Representation

For digital representation of analog data the minimum mean-squared-error criterion is commonly used as a criterion for the basis of optimum quantizer design. In this paper we show that in some situations measures other than the minimum mean-squared-error may be more appropriate. For the signal representation problem, it is shown that the meanabsolute-error criterion has theoretical justification, as again for some signal detection problems it is shown that the mean-squared-error criterion is not the most appropriate criterion. INTRODUCTION Because of the widespread use of digital signal processing methods, the conversion of analog data into digital form is a necessary step in many signal processing systems. A commonly encountered need is to obtain a good representation of analog data with a finite number of bits. We will call this the problem of quantizing data for representation. In other applications it is not the goodness of the representation or “fit” obtained which is of prime importance but rather the extent to which some particular feature of the analog data is preserved in its quantized version. For example, if analog data is to be used to detect the presence or absence of a signal in noise, then the quantization should be performed to maintain as much of the separation of the characteristics of the data under the two hypotheses (signal present or noise only present). Most previous considerations of quantization have been based on the criterion of minimizing the mean-squared-error (MSE) between the analog and quantized data. The use of this criterion cannot be theoretically justified in many instances. In this paper we will show that for both signal representation and signal detection applications, other criteria may be more appropriate and justifiable as a basis for optimum quantizer design. It should be noted that this idea is analogous to one developed in a recent paper[1] where the effects of sampling a continuous waveform for a signal detection application is analyzed directly using detection criteria, rather than the criterion of mean-squared reconstruction error. Some of the results discussed in this paper are based on recent published work by the author [2,3]. SIGNAL REPRESENTATION THE MEAN-ABSOLUTE-ERROR CRITERION We will assume that the analog source input S to the quantizer has an even density function f and distribution function F. Thus we will consider symmetric quantizers q, which are described by the positive input transition values 0 < x1 < x2 < ... < xM-1 and levels y1 y2,...yM for 2M-level quantization. We have q(s) = yi for s,(xi-1, xi), where we also define Let R be an absolutely continuous even function which is increasing on [0,4), with R(0) $ 0. A distortion measure DR may be defined by (1) (where the integral is assumed to exist), and the quantizer minimizing DR may be derived easily [4]. The result is an optimum quantizer parameter set defined by the equations (2) (3) where the prime denotes the first derivative of R. Now consider the distribution function Fq of the output q(S) of the quantizer; with g a weight function with the same properties as R, we may define a distance )g between F and Fq by (4) It is reasonable to look for a quantizer minimizing )g for a given g; this is because q(S) is completely dependent on S. We proceed to do this by first expressing ) as